Beiträge von el_pedriyo

  • Which filesystem

    Hello guys,


    So I have been looking around about some information, like how does ZFS works, UNRAID, snapraid, and so on. And I got to the following conclusion:


    I want to build some sort of UNRAID solution with OMV, which means, I will need for sure mergefs/UnionFS to loadbalance my data through my disks. ZFS was discarded for lacking of this technique, meaning that it makes its own pool, and then it starts stripind the data in all disks, if you chose some sort of RAID.
    Regarding SnapRaid, I am still considering the point of using or not using it. Why? The case, is that probably I won't need it, as I won't be having a parity disk.
    So at the very end; I will have the following:


    2 x 4TB in RAID1
    4 x 8TB in "RAID0" - it is not completely RAID0 as at the very end the data will be load balanced with mergefs


    Up to this point, I need security for my data, so:


    Redundancy: It is already covered as per usual offsite backups, thats why of not using parity disk at the moment.
    File integrity: Here is where it comes, wether to decide to use or not Snapraid, I know Snapraid principle is to "backup" data in a parity disk, and protect it, by running scrubs of it. As I won't need parity, and just scrub option, using Snapraid, is maybe like killing a bird with a granade.
    So, apart from https://github.com/ambv/bitrot that recommended already @Adoby , do you know any other tool to cover data integrity just for an ext4 mergefs filesystem?
    In case of using bitrot solution, do you have any clue, that in case any corruption is detected, I will know which file was corrupted? So that I can restore it directly from the backup.
    Defragmentation: Do you know if OMV does have this option? Or if it is really needed?
    Other protection: Would you think about any other data protection I might need, or you might thing about, that I could be missing?


    Please let me know,


    Kind regards

  • Which filesystem

    pffff, thats quite a pain, though I appreciate the help :D I will take a look at it



    Zitat von jollyrogr

    Good to know you have backups. Because if you're using RAID0 and you lose 1 drive, you lose everything.

    Yes, thats the point of it. But I now the protection I am looking for is to maintain healthy all my files. I do not want them to get corrupt at any point.


    Will research some info about all the filesystems we have been talking that have and doesn't have file corruption detection and will let you know.


    Kind regards

  • Which filesystem

    Hello,


    The only reason I was considering RAID0 was for the speed it will give me for my media disks. I have external backups for all that data, so in case I lose 1 drive, there is not much hassle to recover it.
    But what I was asking now is, in case I do not want to use snapraid, for example for personal data, and use xfs or ext4. Does those filesystems, have scrub option, or any other anticorruption method?


    Please let me know.


    Kind regards

  • Which filesystem

    Yep, that is what I was looking for. But in case I select ext4, what tools do you use to check the consistency of the data? Up to what I know there is no scrub, right?



    Zitat von jollyrogr

    EXT4 with snapraid has worked for me for several years now.

    hmmm, I see.... so you use ext4 and then snapraid, which seems to give some data protection, but in case I want to use RAID0 to strip data accross disks, and not make an unraid solution with snapraid, is there any way to add extra security like the checksum and scrub that snapraid gives?


    Kind regards

  • Which filesystem

    Zitat von ness1602

    ZFS uses scrub technique, which is set in OMV once a month. This means that OMV(ZFS) goes through the disks and finds and repairs if possible those blocks that are damaged ,eq:

    Hello,


    Yes, I currently know about the scrub utility in Freenas, but by coming to openmediavault, what I want is to avoid using ZFS here, so I think scrub option for xfs or ext4, might not work. That is why I was asking for a solution about detecting corrupted data when using ext4 or xfs.



    Zitat von raulfg3

    search forum about snapraid:



    But my suggest is that use ZFS and read more about how ZFs works

    Wow, I guess that is somehow, unraid works, doesn't it?


    That is quite interesting for the media pool I want to make, but in case I just want to make a RAID0, or go ahead with no redundant hard drive?


    I mean, maybe setting up a RAID0 with EXT4 or xfs. I know I might lose all my data if I lose 1 drive, but that is for what I have the backup for. The real issue I am facing on here, refers to data corruction, with other filesystems apart from ZFS, and how in case my data is being corrupted, may I know which one is corrupted, and why it was corrupted, and so on.


    Kind regards

  • Which filesystem

    Hello,


    I just wanted to ask for a bit of help on here:
    I am currently testing various NAS opensource softwares around the place, in this case OMV, and I am still keep understanding what file system should I use.
    My home NAS build, will just be a setup for personal data (photos, music, documents, etc) expecting it to be on 2 drives as RAID1 and then another pool of data for movies/tv shows, in another pool of 4 disks I would say.Now the problem I am facing, is that I have read all over the internet that ZFS has correction of errors when written the data, as it is constantly reading what it writes, to check that it was written correctly, as far as I understood. Also this has a con, which is a lot of memory use.I was trying to find an alternative, in this case OMV, which uses other filesystems as ETX4 or XFS, but I have been researching for data according to this 2 filesystems, and found that they do not use that amount of memory, but that they are not designed to prevent data to be corrupted.
    So the issue in here is: Is there any way or manual script that can be executed for this 2 PMV filesystems to check if that was corrupted somewhere over the disk? I mean, I know about SMART for checking the disks. But in case something else fails, like maybe the RAM, or whatever, when would I start knowing that my data is being corrupted?
    I am currently asking this, because what I do not really want is to through to my NAS 10TB of personal data, and after a 1, 2 or 10 years, start seeing that half of my data is corrupted.
    Also, even if I have a backup, I do not really want to start copying corrupted data, as the backup will turn into a non sense.


    Hope I was clear, and hope someone will be able to through a bit of light on this topic :D


    Kind regards