EXT4 16 TB Limit Question

  • I currently use OMV, but have lately been looking at alternatives such as freenas and such. Problem is, I don't really want to invest in a HUGE amount of RAM or CPU power at the moment to really take advantage of those options. Reason I was looking at switching is because of the 16 TB limit. I am still a bit confused by this. From my research I see it has to do with Squeeze and s2fsprogs 1.41.


    I currently run OMV 4.35 and I see there are newer versions, but I don't think they will fix my issue as they are still running squeeze. I do see in the roadmap that version .6 should be moving away to Wheezy. Will this remove the 16 TB limitation? I currently have a 7x3 TB RAID Z2 array, but I want to grow it to about 10 or 12 discs, which would put me much over the 16 TB limit. I can hold off on this for a little while, but space will become an issue within a few months :(.


    I do not know much about all the different types of filesystems out there, and currently I have enough harddrive space on other computers to relocate my NAS data and do a clean format of my nas and set up a different file system that will support larger than 16 TB volumes, or would this even matter? Is there another option that OMV has that will support higher size limits such as XFS, or will this have the same issue? This nas is for storing multimedia, mostly movies, tv shows, videos, pictures and such. I don't need lightning speed from it, I would just like to saturate a gigabit ethernet port, which I currently do with EXT4.

    • Offizieller Beitrag

    omv 0.6 will be based on Wheezy which will remove this limitation on ext4 but won't be out in your time frame. xfs does not have this limitation now. I would recommend switching to xfs and don't think you would notice a difference.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Do you happen to know the size limit on XFS? Should I worry about reaching it? From wikipedia, the limit is on the order of exibites, is this the same in OMV? I don't want to get stuck again by not being able to upgrade. Also, will XFS allow me to grow the volume as needed, via adding one disc to my RAID Z2 array at a time?


    One more important question, XFS doesn't have the huge overhead like ZFS does it? This is why I don't want to go to freenas as I am worried my 8 gigs of RAM and core i3 are not up the job.

    • Offizieller Beitrag

    You won't hit the xfs limit even on OMV :) Yes, you can grow xfs as well. xfs doesn't have any more overhead than ext4. They are both journal file systems. ZFS is much more. An i3 and 8 gigs of ram are more than enough for any Linux filesystem.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • You could still stay on ext4, but isnt the limit just that OMV cant create filesystems above 16TiB? So grab a parted magic or gparted disc and grow the filesystem with it.


    Besides: XFS has a lower journal than Ext4 has.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Zitat von "davidh2k"

    You could still stay on ext4, but isnt the limit just that OMV cant create filesystems above 16TiB? So grab a parted magic or gparted disc and grow the filesystem with it.


    Besides: XFS has a lower journal than Ext4 has.


    Greetings
    David


    I am pretty noob when it comes to file-systems, and honestly linux in general. What does it mean that XFS has a lower journal? Also, how would I go about growing with a gparted disc?


    Also, most of my files are large files, the vast majority of files are over 1 gig each, if this matters performance wise when deciding which too choose.

  • Bigger files? Even better for XFS from what i've heard and I'm using it myself because most of my files are big files.


    From what I can tell, the Journal of XFS must be lower. I had once two same disks, one partioned with xfs, one with ext4. The one with xfs had way more space available.


    Erm, you know how to add the disks to the raid? Then you just need to adjust the filesystem in parted magic (you can simply use your mouse and drag the filesystem marker to the end, even a 10 year old could do that).


    You can see bechmarks of my raid when you look at my NAS. See my signature for a link. 500MB/s+ both read and write.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Great! Seeing as my files are large, I will probably change my file system to XFS before it is too large to offload to other harddrives. Right now, I think I can just fit it all onto other random drives laying around the house. Also, is there an option to compress data in OMV with XFS? That could be useful once I get my XFS file system up, saving space would be great!

    • Offizieller Beitrag

    You could enlarge the ext4 filesystem with gparted but OMV's utilities couldn't fsck it. So, i wouldn't recommend doing that. xfs is the way to go. Not sure xfs has it but compression wouldn't help much with movies, music, and pics.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I don't know of this particular option.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Although I am nothing of an filesystem expert, I think of xfs as the more stable filesystem. It is the oldest journaling system and was developed more gradually compared to ext. Also NASA is using it! ;)
    But one thing to consider, if you simply cant make backups of such a big volume: It is way more difficult to recover deleted files of xfs, than of ext3. (dont know about ext4 ). Actually if you do a search on it you will mostly hear that it is not possible at all. But nas systems, and therefor xfs installtions, became more and more popular. so now recoverysoftware got better in dealing with xfs. still: consider your files as gone forever, after you delete them (excidentally).

  • @sektion


    You may want to take a look at my NAS thread.
    I recovered 4,5TB of files because of a broken xfs filesystem.


    However I don't know anything about accidently deleted files.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Guys, check my NAS thread before you do any assumptions. It's all written over there. Nothing to be scared of XFS...


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • sounds like an interesting read, will do so later.

    Zitat

    When you say your XFS files system broke, what do you mean? This isn't like, a normal occurrence right? Don't want my data just disappearing on me.


    :D not at all. xfs is a very reliable filesystem.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!